Automatic Recognition of Multiple Brands in Images on Mobile Devices

Visibility of the product on a shelf is an important task of modern marketing principles. Very often companies have agreements with merchants that particular product will be visible and cover defined percentage of the shelf. This trivial task of counting the amount of products or branding logos that are visible within a certain range, is performed manually by an auditor that checks if the agreement is fulfilled. Up till now there does not exist an easy, mobile mechanism that allows to easily capture, recognise and count defined, multiple objects that are visible in the surroundings of the user. Such scenario however, can be achieved using modern mobile phones and their cameras to capture surroundings, and then use their computing power to perform the recognition and counting. For this purpose, feature detectors (such as SIFT, SURF or BRISK) are utilised to create a database of products box images and extracted keypoints are stored. In a new image keypoints are found using the same feature detector, but to avoid problem of multiple identical keypoints, the image is divided and analysed using a sliding window. Keypoints from a window are extracted and are considered as candidates for keypoints that correspond to training images. When enough points are found then perspective transform is calculated. If detected corners are correctly shaped then product is marked with recognised class. In this paper preliminary results of a mobile framework that allows recognition and counting of visible products in surroundings of the user will be presented.